We thank reviewers for thoughtful suggestions which we'll add to improve our paper and are glad they find this work From limited data, it's impossible for any method to accurately estimate nonparametric GIB-1,5,10 all greatly improve BASE models. We'll clarify Gibbs' mixing rate may slow for highly-correlated features (Wang We'll clarify consistency is not referring to our self-attention model, but As suggested by R1, we'll clarify: (1) per-feature conditional distribution is just univariate mixture of Gaussians, not Differences should be statistically significant where p < 0 .05 . As suggested by R2, we'll fix typos issues, use clearer captions, and clarify: (1) In Table 1: Rank is computed by We study how to improve upon latency of ensemble predictors while preserving their accuracy. Thus it is practically performant, and it is very broadly applicable. We'll clarify distillation is not AutoML tool's ensemble and any student model type (may be important if user has particular inference-accelerator / We don't know any accurate AutoML system for tabular data that offers cascades.

Most machine learning (ML) algorithms are highly configurable. Their hyperparameters must be chosen carefully, as their choice often impacts the model performance. Even for experts, it can be challenging to find well-performing hyperparameter configurations.

Recent work has started to tackle this automated machine learning (AutoML) problem with the help of efficient Bayesian optimization methods.

They bring two contributions to existing autoML methods: a meta-learning component, in which they use a list of past datasets to warm-start the bayesian optimizer, and a ensemble construction component, which reuses the ranking established by the bayesian optimizer to combine the best methods instead of just using the one found by the bayesian optimizer, for more robustness. They compare their system, auto-sklearn, to an existing autoML system, Auto-WEKA, and find that they outperform it in a majority of cases. They also compare variations of their system without (some of) the two novel components (meta-learning and ensemble construction) and show that the meta-learning component is the most helpful.

In recent years, Artificial Intelligence (AI) and Machine Learning (ML) have grown tremendously in popularity across various industries. From healthcare and finance to retail and automotive, adopting machine learning models has led to significant advancements[1]. However, building machine learning models traditionally requires deep knowledge in multiple areas, such as data preprocessing, feature engineering, model selection, hyperparameter tuning, and evaluation[2]. For many beginners and even experienced practitioners, this process can be time-consuming and technically challenging. This is where AutoML (Automated Machine Learning) comes in. AutoML simplifies the process of building machine learning models by automating many of the steps that would otherwise require manual intervention [3]. AutoML tools can automatically preprocess data, select the most suitable algorithms, and fine-tune hyperparameters to produce highly accurate models [4]. This automation not only speeds up the model development cycle but also allows users without deep knowledge of machine learning to create models with comparable performance to those made by experienced data scientists.

Automated machine learning (AutoML) was formed around the fundamental objectives of automatically and efficiently configuring machine learning (ML) workflows, aiding the research of new ML algorithms, and contributing to the democratization of ML by making it accessible to a broader audience. Over the past decade, commendable achievements in AutoML have primarily focused on optimizing predictive performance. This focused progress, while substantial, raises questions about how well AutoML has met its broader, original goals. In this position paper, we argue that a key to unlocking AutoML's full potential lies in addressing the currently underexplored aspect of user interaction with AutoML systems, including their diverse roles, expectations, and expertise. We envision a more human-centered approach in future AutoML research, promoting the collaborative design of ML systems that tightly integrates the complementary strengths of human expertise and AutoML methodologies.